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The Predictive Value of the Watson-Barker Listening Test 



Using the Watson-Barker Listening Test Form A and Form B as 
pretest and posttest at the Freshman level, this research attempted to 
determine whether the two forms of the test and the differences between 
pretest and posttest would be significant predictors of students' final 
GPA. The results indicated that, while each test form had significance 
against GPA scores, the correlation was not high enough to predict final 
GPA, and the difference between pre- and posttest had no predictive 
value when correlated with GPA. 



Introduction 



In the May 1 982 Listening Post, McKibben reported on the use of the Brown-Carlsen 
Listening Comprehension Test to evaluate the listening skills of students enrolled in the 
College of Basic Studies at the University of Hartford. McKibben noted that one 
direction of future research would be 

a study of the most significant indicators of academic success in our Basic 
Studies program. We are correlating the Brown-Carlsen Listening 
Comprehension Test scores with Nelson-Denny Reading Test vocabulary, 
comprehension, and total reading scores, rank in high school class, SAT 
mathematics and verbal scores, and secondary school Carnegie Units of 
Instruction (as an indicator of background). We are attempting to determine 
which of these factors~or which combination of factors— provide an "early 
warning system," i.e., predict which students may either be dismissed for 
academic reasons or be placed on probation. 

In a subsequent summary of continued research, McKibben (1983) reported that the 
use of the Brown-Carlsen Listening Comprehension Test successfully classified 65% of 
the students "at risk", while the Nelson-Denny Reading Test was a successful indicator of 
60% of the cases. McKibben concluded that "listening scores seem to be slightly better at 
predicting group membership." 

A search and review of the literature has failed to locate any continued use of the 
Brown-Carlsen Listening Comprehension Test or any other listening test to predict 
academic failure or success, yet it appears reasonable to expect some relationship 
between listening skills and subsequent academic work. 
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Communication educators have long recognized that the basic skills in speaking and 
listening are necessary for successful completion of college studies and future success in 
one's selected vocation. Utilizing surveys conducted in business, such as that reported by 
Roos (1984) in the Des Moines Register and the comments of corporate leaders such as 
Iacocca (1984), educators have frequently used the need for skills as the rationale for 
speaking and listening training during the first year of college. Presumably the college 
student will use listening skills in all college courses, and the proper use of those skills 
will result in improved knowledge, understanding, comprehension, and application of that 
knowledge in college and on the job. However, if the rationale is correct, then one could 
expect that listening tests could be used to assess the teaching/leaming of listening skills 
as well as predict the final success of the college student if listening tests measure skills 
needed to succeed. In fact, Watson, et al.( 1 991 ), maintain that the Watson-Barker 
Listening Test (WBLT) can be used as "a valid, reliable listening measurement tool 
to.. .identify listening skills that need improvement... [and to] assess the effectiveness of 
various instructional strategies.... "(p.3) However, the need to identify improvement and 
assess effectiveness of instructional strategies assumes that the need for the improvement 
of listening is needed, at least, for subsequent academic work. Wolvin and Coakley 
(1994) state that "communication educators recognize competence in communication as a 
major focus both in the curriculum and in the assessment of the knowledge, behaviors, 
and attitudes that we work to develop with our students. "(p. 148) 

Given that assumed stance, Wolvin and Coakley have compiled a number of 
competency and skills lists, which range from the worker skills of Muchmore and Galvin 
to the comprehension competencies of the Wingspread Conference to listening 
components, but they had not completed a comparison of those lists. Wolvin and 
Coakley concluded that the major research questions include "How do current lists of 
characteristics of listening competency differ? How are they similar?" (p. 158) 

A cursory, and somewhat intuitive, examination of the lists of skills and/or 
competencies identified and analyzed by Wolvin and Coakley indicate several basic skills 
implied by some or all. For example, to "recognize main ideas" (p. 153) from the 
Wingspread Conference assumes that the listener can also attend to and note the idea, the 
basic information, similar to "obtain necessary information" from the Muchmore and 
Galvin list (p. 153). To "discriminate between statements of fact and statements of 
opinion" assumes obtaining the needed information to know, to recognize opinion 
statements (p.152.) The Wingspread Conference list also includes "distinguish between 
facts and opinions" (p. 153). To "listen to comprehend the content of messages and the 
intent of speakers" from the Maryland list of goals (p. 155) assumes that the listener can 
first recognize information, file that information, and then manipulate that information. 
For college students as well as corporate employees, following directions (see p. 153) is 
another important skill in listening, which serves as the foundation for understanding the 
expectations of others (Muchmore and Galvin) and being able to respond to others' 
requests (Rhodes.) Binford (as quoted by Fitch-Hauser and Hughes in 1987) completed 
an analysis of 25 "different tests and found. ..that those tests supposedly covered 22 
different listening skills" (p.132.) That list included the ability to follow directions and to 
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remember important details which would assume listening for and filing into memory, 
understanding meanings, and judging mood (see p. 132.) 

Many of the skills common to the lists of skills and competencies discussed by 
Wolvin and Coakley and the skills compiled by Binford appear to be important skills for 
academic study and potential academic success. College students would be expected to 
listen for and to follow directions in order to complete assignments. They are required to 
attend to speakers/instructors, to detect and file in memory important details from lectures 
and discussions, to select and organize main ideas, to understand the implications and 
emotional impact of others' words. In short, it appears that college students need virtually 
all those skills and competencies to complete their academic work. It is that similarity of 
skills considered necessary for academic success that McKenzie and Clark (1995) 
consider to be the overwhelming direction of the cognitive dimensions of present 
listening tests. 

If the focus of present listening tests is cognitive, then one could hypothesize that, 
since so much of the academic work at the college level is cognitively related, a listening 
test might be testing the same skills needed to succeed at college work. While no one test 
exists that would measure all cognitive skills needed and no test exists to test all sub- 
skills of listening, the congruence of the WBLT to those skills that appear to be common 
among the listening skills and competency lists could make it a viable test to predict 
success in academic work. Those listening skills common to the lists of skills included 
listening for information and filing that information in short term memory, listening to 
and following directions, listening for the implications and emotional character of others' 
statements, interpreting others' meanings. The WBLT claims to measure the same skills. 
It "measures the listener's skill in interpreting message content.. ^understanding the 
meaning of conversations..., remembering lecture information..., interpreting emotional 
meaning..., [and the] listener's ability to follow instructions."(Watson, et al., 1991, p. 2) 



The research study 

Dordt College is a small, private, four-year liberal arts, Christian college of 1300 
students located in Northwest Iowa. The majority (approximately 75%) of its students are 
traditional students from private, Christian high schools distributed around the United 
States and Canada, however, over half (52%) come from Iowa, Minnesota, Nebraska, and 
South Dakota. 

All students are required to take the one-semester, three credit speaking and listening 
course, and most students complete the course during their freshman year. The course 
begins with the WBLT Form A. Within the first two weeks of the course, the students 
study and discuss a chapter on listening including its value, the types of listening, and 
how to listen. During the course, students will complete two listening exercise tapes 
which also supply listening techniques, and the students will complete two sets of 
exercises during speeches given in class. The first exercise consists of listening to 
selected speeches and then completing a form which requires the speaker's purpose and 
main points. The second exercise, also during class speeches, requires students to 
recognize and record selected claims, the evidence to support those claims, the source of 
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the evidence, and the testing of that evidence. The final listening exercise is a speech 
criticism. At the end of the course, the WBLT Form B is administered as a posttest. The 
listening exercises during the speeches are graded. All grades and scores are recorded. 
The grades and scores of the 1997 BA degree graduates were collected to test the 
hypotheses of this research. 

Given the commonness of listening skills and some of the skills important for 
academic work, one could expect that scores from WBLT would correlate with academic 
scores measured as Cumulative GPA. 

Hypothesis 1— 

The scores on the Watson-Barker Listening Test Form A and Form B will be 

significant predictors of GPA. 

If the WBLT is a significant predictor of academic success, then the improvement 
demonstrated from pretest to posttest following training would predict higher academic 
grades assuming that the pretest and posttest are sufficiently similar. The WBLT has had 
means and standard deviations calculated on large, national populations. Since the 
difference in means between Form A and Form B is so small (Form A— 58.8 and Form B- 
-58.7) and the standard deviations demonstrating a decrease from Form A (7.6) to Form B 
(6.9) (Watson, et al., 1991, p. 13) then one would expect that an increase from pretest to 
posttest would correlate with GPA. The increase in score from Form A as pretest to 
Form B as post-test will be a significant predictor of academic success. 

Hypothesis 2— 

The difference in score from Watson-Barker Listening Test Form A to Watson-Barker 

Listening Test Form B will be a significant predictor of GPA. 



Procedure 

The population selected for this research was the students who graduated with BA 
degrees in May, 1997. While the college also has an Associate of Arts degree, those 
students were not considered part of the population in order to maintain a homogeneous 
group. Those students who had transferred to Dordt College after their Freshman year 
and had not taken the Fundamentals of Speaking and Listening course were also 
excluded. The result was 190 students included in this study. The WBLT Form A and 
Form B scores were collected from the instructors. Additional grades (those of the two 
listening exercises) were also collected to test their predictive value, but those tests are 
not included in this report. The final, cumulative GPA for each graduate was collected 
from the Registrar's office using two different sets of identification numbers to guarantee 
confidentiality. 

The WBLT scores of Form A and Form B and the student GPAs were tabulated to 
determine maximum and minimum scores, means, and standard deviations. See Table 
#1. The scores for Form A and Form B were each related to student GPA, and the 
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difference between Form A and Form B was also related to student GPA using Pearson 
correlation. The significance levels using a 2-tailed test were calculated. See Table #3. 



Results 

The correlation values are found in Table #3. The significant correlation between 
Form A and Form B seems to indicate that Forms A and B are measuring skills which 
supports reliability. The significant correlations between Form A and GPA and Form B 
and GPA seem to indicate valid measurement of some of the same cognitive skills as 
those needed for successful academic work when academic work is measured by GPA. 
The significant relationships of Form A and Form B to GPA would appear to support the 
first hypothesis— the scores on the WBLT will be significant predictors of GPA, but when 
calculating the GPA variance accounted for (indicating the accuracy of prediction) by 
Form A and Form B with GPA (r x 100), Form A predicts 6% of the variance in GPA, 
while Form B predicts 8% of the variance in GPA. 

If the scores on the WBLT were significant predictors of GPA, then one would 
anticipate that an increase in scores from Form A to Form B would also serve as a 
predictor of GPA because an improvement in those listening skills associated with 
academic work should indicate improved GPA. The test of the difference from Form A 
to Form B related to GPA failed to be significant. This test failed to support the second 
hypothesis— the difference in score from WBLT Form A to WBLT Form B will be a 
significant predictor of GPA. However, the correlation of Form B to GPA was higher 
than the correlation of Form A to GPA. 



Table #1 

Genera] Population Statistics 





N 


Minimum 


Maximum 


Mean 


Std. Dev. 


Form A 


190 


36.0 


94.0 


66.153 


9.429 


Form B 


190 


.0* 


98.0 


81.558 


9.712 


GPA 


190 


2.1 


4.0 


3.292 


.475 



* One student in the population did not take Form B. Otherwise the minimum would be 58.0. 

Table #2 
Gender Statistics 



Male N Form A Form B GPA 



Mean 


93 


65.785 


81.484 


3.155 


Std. Dev. 
Female 




10.448 


8.182 


.460 


Mean 


97 


66.505 


81.629 


3.424 


Std. Dev. 




8.377 


11.024 


.454 
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Table #3 
Correlations 





Form A 


Form B 


GPA 


Form A/B Dif. 


Form A 


1.000 


.323** 


.251** 


— 


Form B 


.323** 


1.000 


.294** 


— 


GPA 


.251** 


.294** 


1.000 


.044 


Sig. (2 -tailed) 
Form A 




.000 


.000 




Form B 


.000 


— 


.000 


— 


GPA 


.000 


.000 


— 


.543 


**. Correlation is significant at the 0.01 level (2-tailed) 


N=190 





One possible explanation for the failure to support the second hypothesis is that the 
amount of variance accounted for in GPA by Form B is low enough (8%) that it is 
colored by the remaining variance of GPA. A second explanation might be that students 
with higher GPA attained high scores on Form A and the difference between their scores 
on Form A and Form B was not as great as the difference between scores for students 
with lower GPA. 

The population that was tested could be a factor in the results and might lend a 
possible explanation for the failure to support the hypotheses. A review of the statistics 
for the general population (Table #1) indicates some differences from the reported means 
and standard deviations for the WBLT. The means for both Form A and Form B are 
considerably higher for the tested population than the population used by Watson et al. 
The WBLT means are listed as 58.8 and 58.7, while the means for the tested population 
were 66.15 and 81.55 for Forms A and B respectively. The mean for Form B of the 
tested population is 22.8 percentage points higher than the mean reported by Watson et 
al.(l 991 .) The standard deviations for the tested population also differ from those 
reported by Watson et al. The standard deviations reported by Watson et al. are 7.6 and 
6.9 while the corresponding standard deviations for the tested population are 9.429 and 
9.712 respectively. While the means are considerably higher for the tested population, 
the standard deviations are also larger. One might conclude that the high test scores and 
the size of the tested population (N=190) have actually contributed to the significance 
levels while not yielding higher correlations, particularly for the correlation of the 
difference between Form A and Form B with GPA. This explanation, however, does not 
recognize that the tested population was tested with itself and not against a different 
population. Any increase in scores would be within the population. 

An examination of possible differences in gender also fails to supply an explanation 
for only moderate support of hypothesis 1 and failure to support hypothesis 2. (Table #2) 
The population is balanced for gender. The means scores for males and females on both 
Form A and Form B are very close. The only variation of any note is the difference in 
GPA between male and female, but that difference is not significant. 
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Past education (elementary and high school) would not likely be a factor. In my 
research conducted in the 'feeder' schools, little or no teaching of listening was occurring 
(Vander Kooi, 1978), and subsequent informal checks on students' past training in 
listening indicate no change. I believe that one can conclude that past education of this 
population is no different from that of the general population in the training of listening. 

It appears that the character of the WBLT, both Form A and Form B, are sufficiently 
alike and measure the same set of skills that are common with some basic academic skills 
to be significantly correlated with GPA at the .251 and .294 levels, but the predictive 
values of the correlations are low and any change in scores from Form A to Form B do 
not correlate with GPA. 



Discussion 

For years Communication educators have advocated increasing listening skills and 
have appealed to comments from business persons and other educators as the rationale for 
training in listening. For years those same Communication educators have accepted the 
grounds that better listening will result in better relationships, better knowledge, and a 
better education. They have also recognized the role that motivation plays in listening. 
There is, therefore, a particular appeal to knowing that attaining a good grade in listening 
will predict good grades in other classes. One could even consider it a good step in 
motivating students, if he/she could say that a particular test will predict future grade 
point average and that after some study and training, an increase in a test grade will show 
that better grades are attained. One might conclude that the results of this study do allow 
the conclusion that this particular test, the WBLT, does significantly predict a percentage 
of future GPA. The context of this research gives some credence to the conclusion. Both 
Form A and Form B were administered in the first-year college speaking/listening course. 
The tests were not graded so the results had to depend upon the student's personal 
motivation. Training and the study of listening occurred between the two tests and the 
scores increased dramatically. Those test scores were correlated to the final GPA of 
students who graduated three or three and one half years after taking the first-year course. 

While Communication Educators might wish to increase student motivation by 
reporting a significant correlation between listening test scores and GPA, there are 
reasons to question that use of current research information. Fitch-Hauser and Hughes 
(1992) recognize that listening tests have "fallen short of that [reliability and validity] 
needed to develop a conceptual underpinning for listening."(p. 8) In an earlier study they 
attempted a factor analysis of four listening tests including the WBLT and concluded that 
"these tests are not homogeneous. That is they are testing something in addition to 
listening." (1987, p.146) The same conclusion could be made about the results of this 
research. The WBLT could be predicting a limited percentage of the variance in GPA, 
but it appears to be doing so because it is measuring some of the same skills that are 
required by GPA— those skills might or might not all be listening skills. For example, the 
WBLT, as do many listening tests, assumes a common vocabulary of the verbal and the 
nonverbal. The nonverbal vocabulary is most evident in the section which tests the 
meaning of conversations, Part II. An informal research conducted by another member 
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of the Communication Department at Dordt College has found that Part II of the WBLT 
has the lowest scores on Form A which coincides with the scores reported by Watson et 
al. (1991), but those same students show the greatest amount of improvement in that 
same section on Form B— beyond the mean reported by Watson et al. I suspect the 
difference is nonverbal vocabulary. Informally, students report greater difficulty with 
accent with Form A than with Form B. Other skills might also be in play— skills that are 
necessary for successful academic work. Those skills might or might not be the same as 
the skills considered listening skills. The congruence between a package of common 
skills in the WBLT and those needed for academic work might be an explanation for the 
significance levels of the correlations to support hypothesis 1 and the lack of support for 
hypothesis 2. In other words, whatever the WBLT is evaluating, it is evaluating some, 
albeit a low percentage, of the same set of skills needed to gain a higher GPA, whether or 
not those skills are listening skills. 

With Fitch-Hauser and Hughes (1987), as well as others, the results of this research 
call for continued analysis and research on the validity of listening tests beyond face 
validity and validity found in a factor analysis of a variety of listening tests. The goal is 
still there-be able to demonstrate that improvement in listening skills as demonstrated in 
a standard test will result in a greater likelihood of academic success. 

However, future research should also extend beyond that of skills, if a listening test is 
to predict academic success. As Wolvin and Coakley (1994) note-"listening competence 
must...extend to attitudes and knowledge" -ipxesumabXy a knowledge of the 
communication content (p. 153.) Wolvin and Coakley also note that "the development of 
listening attitudes must be part of any solid listening education" (p. 153.) I would assume 
that motivation would be an important part of that attitude training. I have observed 
students who had low WBLT scores attain high GPAs. Their motivation and work ethic 
appeared to account for high achievement during college studies. 

Until the time that a clear understanding of the relationship between the WBLT and 
GPA is attained, Communication educators can still use listening tests to assist in analysis 
of listening and to ascertain, within the scope of the individual test, what improvements 
might be made by students from pretest to posttest. We can use the results circumspectly, 
while continuing to push for theory development and test validity beyond what we now 
have. However, I believe that attempting a prediction of academic success based on 
listening test scores is hazardous. Communication educators will have to continue to rely 
on employer surveys and the logical associations of skills to justify teaching listening at 
the college and university level. 
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